Measurement Error , Legalized Abortion , the Decline in Crime : A Response to Foote

Donohue and Levitt (2001) argue that the legalization of abortion in the United States in the 1970s played an important role in explaining the observed decline in crime approximately two decades later. Foote and Goetz (2005) challenge the results presented in one of the tables in that original paper. In this reply, we regretfully acknowledge the omission of state-year interactions in the published version of that table, but show that their inclusion does not alter the qualitative results (or their statistical significance), although it does reduce the magnitude of the estimates. When one uses a more carefully constructed measure of abortion (e.g. one that takes into account cross-state mobility, or doing a better job of matching dates of birth to abortion exposure), however, the evidence in support of the abortion-crime hypothesis is as strong or stronger than suggested in our original work. Donohue and Levitt (2001) put forth the hypothesis that the legalization of abortion in the United States in the 1970s played an important role in explaining the observed decline in crime approximately two decades later. In that paper, we present a variety of types of evidence in support of the argument: (1) a calibration exercise based on pre-existing estimates of the impact of abortion and the distribution of women seeking abortions, (2) patterns in the national time-series data, (3) a comparison of the crime paths of early-legalizing states and those in which abortion only became legal with Roe v. Wade, (4) differences in crime patterns in states with high and low abortion rates after abortion became legal everywhere, (5) the fact that reductions in crime in high-abortion states were limited to those born after abortion legalization, and (6) arrest patterns by single year of age that are negatively related to abortion exposure. The findings of each of these analyses are broadly consistent with the hypothesis that legalized abortion reduced crime through a combination of smaller cohort sizes and lower criminal propensities among those born after legalization. In a widely circulated working paper, Foote and Goetz (2005) challenge the sixth piece of evidence described above (namely, that arrest patterns by single year of age are consistent with abortion reducing crime). Foote and Goetz (2005) present two strands of evidence in making their argument. First, they correctly note that the text of Donohue and Levitt (2001) reports that state-year interactions are included in some of the columns in Table 7 of the paper, but that the published version of the paper inadvertently omitted 2 Foote and Goetz (2005) does not address any of the five other analyses in Donohue and Levitt (2001). Joyce (2004) is the only published paper we are aware of that challenges our initial findings. Donohue and Levitt (2004) contains our response to that earlier paper. these interactions. When state-year interactions are added to the specification, the parameter estimates shrink, but remain economically large and statistically significant. Second, Foote and Goetz (2005) shows that when one includes state-year interactions in Table 7 of Donohue and Levitt (2001) using a data set that we prepared for Donohue and Levitt (2004) and changes the dependent variable from the log of arrests to the log of arrests per capita, the coefficient on the abortion variable becomes small and statistically insignificant. Based on this result, they conclude that legalized abortion may have reduced crime because of smaller cohort sizes, but that individuals exposed to legalized abortion in utero are no less criminal on average. In this reply, we address in turn the two issues raised by Foote and Goetz (2005). While it is with great embarrassment that we acknowledge that state-year interactions were omitted from four of the eight regressions in the published version of Table 7 of our original paper, the mistake in the table, as we show, has a relatively minor impact on the results. With respect to the second challenge raised by Foote and Goetz (2005), we show that the absence of effects when including state-year interactions and using per capita arrest rates is an artifact of the combination of a very crude abortion proxy and empirical specifications that remove an enormous amount of the true signal in the data by controlling for state-age interactions, age-year interactions, and state-year interactions. When building on our work in Donohue and Levitt (2004), we more carefully construct the abortion measure so that it (1) better corresponds to the actual month and year of birth 3 While any mistake is embarrassing, we are at least glad to have facilitated the catching of this error by posting our data and do files on the web at http://islandia.law.yale.edu/donohue/pubsdata.htm. Jonathan Gruber, Phillip Levine, and Douglas Staiger were actually the first to discover this programming error in our code; they intend to discuss it as part of a larger analysis of the impact of abortion in a paper that is in process. 4 The same omission of the state-year interactions is also repeated in the bottom two rows of Table 1 of Donohue and Levitt (2004), which used the same code as the original paper. In neither paper did we rely on these flawed regressions to estimate the magnitude of the impact of abortion legalization on crime. of the individual, (2) incorporates cross-state mobility between birth and adolescence, and (3) reflects the state of residence of those having abortions (as opposed to the state in which the abortion is performed), the results we obtain are as strong or stronger than the original results included in Donohue and Levitt (2001). The remainder of this paper is structured as follows. Section I discusses the mistake in the published version of Table 7 of Donohue and Levitt (2001). Section II demonstrates the presence of substantial measurement error in the crude abortion proxy used in our original paper and discusses the construction of a more thoughtful proxy. Section III shows the impact on the estimates of correcting for measurement error. Section IV revisits the question of the channels through which legalized abortion reduced crime. Section V concludes. I. The mistake in the original Table 7 of Donohue and Levitt (2001) The final table (Table 7) of Donohue and Levitt (2001) analyzes arrest rates by state of residence, year, and single year of age. As reported in the text of the original paper, the most saturated models estimated in this section of the paper take the form: 1 ln( ) stb sb sa ta st stb ARRESTS ABORT β γ λ θ ε = + + + + (1) where s, t, a, and b correspond to state of residence, year, age, and birth cohort respectively. The variable ARRESTS is the raw number of arrests for a given crime category. The abortion proxy used is the abortion rate in the current state of residence in the calendar year most likely to have preceded an arrestee’s birth given the year in which 5 Most of the analysis in Donohue and Levitt (2001) uses reported crime statistics from the Federal Bureau of Investigation’s Uniform Crime Reports as the measure of crime. Because the age of an offender is recorded only when an arrest is made, one must use arrest data rather than crime data when carrying out an analysis by age of offender. they were arrested and their age. Included in the specification are state-age interactions, year-age interactions, and state-year interactions. Due to an oversight on our part, which we deeply regret, the version of Table 7 that was published did not include state-year interactions, despite what was written in the text of the article. Table 1 of this paper reproduces the incorrect version originally published as Table 7 of Donohue and Levitt (2001), along with the corrected columns as they should have appeared. Columns (1) (4) reproduces the original results reported for violent crime. Columns (5) and (6) are identical to columns (3) and (4) respectively, except that state-year interactions are included in columns (5) and (6). Adding the stateyear interactions reduces the abortion coefficient from -.028 to -.013, but the estimate remains statistically significant at the .01 level. The abortion coefficient with state-age and state-year interactions is about the same in magnitude as when only state-fixed effects are included (column 1). When arrest rates are broken down by single year of age (column 4 versus column 6), all of the estimates remain negative, although almost always smaller. Adding state-year interactions to the property crime regressions induces a similar pattern of coefficient changes. II. Can measurement error in the abortion proxy explain the smaller estimates obtained when state-year interactions are included in the specification? There are two possible explanations why the coefficient on the abortion variable shrinks when state-year interactions are included. The first explanation is the presence of 6 Note that the numbers in Table 2 do not exactly match those reported by Foote and Goetz (2005) because in addition to correcting the programming error in our original paper, they also used our Donohue and Levitt (2004) data set that extends the number of years covered beyond those included in our original paper. omitted factors that vary by state and year which are positively correlated with current crime rates and negatively correlated with the abortion rate two decades earlier. The second possibility is that there is substantial measurement error in the abortion proxy. As more controls are included in the regression, the remaining variation in the abortion measure may become dominated by noise. The shrinkage of the abortion coefficient, in this scenario, is due to attenuation bias. Presenting definitive statistical evidence one way or the other on the omitted variable explanation is extremely challenging, precisely because the factors involved are not readily observed. It is possible, however, to systematically explore the issue of measurement error in the abortion proxy to determine whether it may be at the root of the diminished effect of abortion when state-year interactions are added to the specification. An examination of the correlation between the Alan Guttmacher Institute (AGI) abortion measure (by state of occurrence) and another abortion proxy independently collected by the Centers for Disease Control (CDC) provides a first indication of how noisy these measures of state-level abortion rates are. The results are presented in Table 2. The raw correlation between the proxies, not controlling for any other factors, is 7 Foote and Goetz (2005) conjecture that crack cocaine might be a possible omitted factor. Crack cocaine emerged in the mid-1980s, peaking in the early-1990s, before falling slowly thereafter (Fryer et al. 2005). In order for crack cocaine to plausibly explain these results, it must be the case that (1) crack cocaine varies by state and year, (2) is positively correlated with crime, and (3) is negatively correlated with abortion rates in prior decades. Crack cocaine, as proxied by the crack index constructed in Fryer et al. (2005), did indeed exhibit substantial state-year variation and is positively associated with crime. On the other hand, the data suggest that, if anything, the places most adversely affected by crack in the 1980s and 1990s were those that had high abortion rates in the 1970s, not low abortion rates. The five states with the highest estimated level of crack in Fryer et al. (2005) were Maryland, New York, New Mexico, New Jersey, and Massachusetts. Except for New Mexico, these high crack states were among the top ten highest abortion states after legalization. In contrast, of the five lowest crack states (South Dakota, Montana, Wyoming, North Dakota, and Nebraska), three of these states were among the ten states with the lowest abortion rates. Thus, the omission of crack cocaine should lead the impact of abortion to be understated, not exaggerated. See Levitt (1999) and Donohue and Levitt (2004) for further discussion of this issue. Although the CDC currently collects data on abortion rates by state of residence and by place of occurrence, as far as we can determine, only the data by place of occurrence are available over the entire period of interest. Therefore, we use that measure as our instrument. .849, as shown in column (1). Thus, in the raw data, these two proxies do indeed track each other closely. In the regression analysis, however, it is not the measurement error in the raw data that matters, but rather the measurement error after partialling out the set of included controls. As Table 2 demonstrates, the more controls that are included, the lower the correlation between the two abortion measures, suggesting that the signal-tonoise ratio is substantially reduced in the portion of the abortion measure that is actually being used to identify the parameter estimate. In the most heavily saturated models that include state-year interactions, the correlation is only .396. What remains of the variation in these two proxies is only relatively weakly correlated, implying that there is substantial measurement error in at least one, and likely both of the indicators. Even the correlations shown in Table 2, however, are likely to dramatically overstate the amount of signal in the abortion proxy used by Donohue and Levitt (2001) and Foote and Goetz (2005) for two reasons. First, given that both AGI and CDC collect their data through similar survey methods, it seems quite likely that the measurement error in the two data series will actually be positively correlated, implying that some of the observed correlation is not due to true signal, but instead to correlated errors in the two proxies. Second, and probably far more important, the crude measure of abortion that was used has other obvious weaknesses. The proxy for abortion exposure in these two earlier papers for individuals of age a in state s in year t is the AGI estimate of abortions performed in state s per live birth in state s in year t-a-1. This particular choice of proxy suffers from three additional sources of measurement error that are not captured in the correlations reported in Table 2: 1) A non-trivial fraction of abortions performed in the United States, especially in the time around when legalization occurred, involved women crossing state lines to get an abortion. As a consequence, measuring abortions in terms of the state in which the abortion is performed, rather than the state of residence of the woman getting the abortion, induces further measurement error into their abortion proxy. 2) Cross-state mobility. Based on census data, more than one-third of Americans aged 15-24 currently reside in a state other than the one in which they were born. For these individuals, one should not expect lagged abortion rates in their current state of residence to matter, but rather, the lagged abortion rate in the state in which they were themselves born. 3) Using the year t-a-1 as the relevant timing of abortion exposure leads to the wrong year of birth for a non-trivial fraction of individuals. The arrest data reports how many 19 year olds are arrested in a state and year for a particular crime. A 19 year old arrested in 1993 may have been born as early as Jan. 2, 1973 (making him 19 if arrested on Jan. 1, 1993) or as late as Dec. 31, 1974 (if arrested on Dec. 31, 1993). Because of the way the arrest data are collected, there is a two-year window of birthdays that is relevant and not adequately captured by the simple abortion proxy. The combination of access to new data series that we were not aware of at the time our initial research was published, along with more careful decisions in constructing the abortion measure, allows us to build an improved abortion proxy that addresses each of these three measurement error concerns cited above. The first source of measurement error in the original abortion proxy is that many women were crossing state lines to get abortions in the 1970s. When calculating abortion exposure of people residing in a state, knowing the abortions performed on women residing in a state is clearly preferable to abortions performed in a state, regardless of where the woman actually lives. When our initial research was published, we were not aware of the fact that the AGI calculates abortions both by location of the procedure (the data we used) and by state of residence of the mother (the more theoretically desirable measure). Given the availability of this better measure, we can eliminate this source of measurement error. In Donohue and Levitt (2004), we argued that this improved abortion measure was clearly preferable to the original proxy we had chosen. In that paper, we demonstrated that correcting this measurement error substantially increased the estimated abortion coefficients. To deal with the second source of measurement error, namely cross-state mobility, we use the 5 percent PUMS sample from the 1980, 1990, and 2000 decennial censuses to measure the distribution of states of birth among current residents of a state at each age. One can then linearly interpolate using the three censuses to estimate the distribution of states of birth in intervening years as well. Rather than using lagged abortion rates in the current state of residence, one can more closely proxy the true in utero abortion exposure using a weighted average of the lagged abortion rates in the 9 The data set we provide to researchers who want to replicate our findings reflects the improvements we made to our approach after the original paper was published, e.g. it includes abortion measures both by state of occurrence and state of residence, and also extends the years covered beyond the original sample. We find it puzzling that Foote and Goetz chose to use the longer data series (which slightly reduces the point estimates) when “replicating” our original Table 7, but did not elect to use or even discuss the better abortion measure (which substantially increases the estimates), in spite of citing Donohue and Levitt (2004) which argues strongly for the improved measure. 10 In Donohue and Levitt (2001), we carry out precisely this exercise in the sensitivity analysis to our primary identification strategy, reported in Table 5. As would be expected, the abortion coefficient became larger when this correction was made. The measurement error problem in that setting is likely to be much less severe than in the analysis of arrests by single year of age. The unit of observation in Table 5 of Donohue and Levitt (2001) is a state-year so the controls are limited to state and year fixed effects and covariates. In contrast, in the corrected Table 7 of Donohue and Levitt (2001), controls are included for state*age, age*year, and state*year interactions, as well as all the main effects of those interactions. states of birth of those currently living in the state, with the weights determined by the percent of a state’s current residents born in each state. The final source of measurement error in the crude abortion proxy is a failure to correctly match some individuals to the true abortion exposure they experienced by using year t-a-1 as the date of abortion exposure. Table 3 presents a matrix of the year of likely abortion exposure for those arrested at age 19 in year 1993 under the assumption that births and arrests are uniformly distributed across months, births occur nine-months after conception, and abortions occur at week 13 of a pregnancy. For instance, roughly half of those individuals arrested in January of year 1993 at age 19 will have been born in year 1973. The rest will have been born in year 1972. Arrests made in the middle of the year match well with a year of birth of 1973. Towards the end of the year, however, an increasing fraction of those arrested are actually born in year 1974, rather than 1973. Totaling the fraction of all births to this cohort, 25 percent fall outside of 1973. To better capture the actual abortion exposure of this cohort, we construct an improved abortion measure that is a weighted average of the abortion rates in years t-a-2, t-a-1, and t-a, with the weights corresponding to the fractions in Table 3. Even after making these three corrections described above to the abortion proxy, measurement error may still be a problem given imperfections in the way that abortion data are gathered. A standard approach for dealing with measurement error of this sort is instrumental variables. If one has two noisy abortion measures, but the measurement error in the two proxies is uncorrelated, instrumenting for one abortion proxy using the other will eliminate attenuation bias. In this setting, the CDC’s independently generated 11 One could also be less parametric and simply include separately the abortion rates in these three years in the regressions and total up the combined impact of all three years of abortion exposure. The estimated effects are very similar, but slightly larger, when we run the specifications non-parametrically. measure of legalized abortions is likely to be an excellent instrument. Because there is so much noise in each of the measures, the standard errors increase when doing this IV procedure, but under a standard set of assumptions, the estimates obtained will be purged of the attenuation bias that will be present due to measurement error. III. A comparison of estimates with and without corrections for measurement error Table 4 presents a comparison of the results before and after the corrections described above are made to deal with measurement error. The top row of the table reports the estimates using the uncorrected measure for violent crime arrests; the second row shows results when the three corrections noted above are made to the abortion proxy. The third row uses the corrected abortion measure, and in addition instruments for the AGI measure using the CDC measure. In all cases, the abortion coefficient jumps sharply when the original measure is replaced with the more carefully constructed measure. Even the estimates from the most saturated version of the model which includes age-year, state-age, and state-year interactions yields a coefficient of -.046 that is highly statistically significant. This estimate is larger than the coefficients reported in the original Donohue and Levitt (2001) paper that did not include state-year interactions. 12 To the extent that there is positive correlation in the measurement error in the two abortion proxies, the instrumental variables estimates will tend to understate the impact of abortion exposure on crime. Correlated measurement error will lead to an exaggeratedly large first-stage relationship between the two abortion proxies, which will result in the second-stage estimates being too small in absolute value. 13 The results in the top row of each panel of Table 4 differ from the results in Table 1 because in Table 4 we follow Foote and Goetz (2005) in using the expanded Donohue and Levitt (2004) data set covering the time period 1985-1998, whereas the results in Table 1 match our original sample of 1985-1996. 14 All three of the corrections we make to the abortion proxy contribute to the increased coefficient. Adjusting for state of residence as opposed to state of occurrence in the model with state-year interactions raises the coefficient from -.009 to -.021. Correcting for cross-state mobility in addition to the first fix raises the coefficient from -.021 to -.039. Getting the birth years right moves the coefficient from -.039 to .046. Instrumenting with the CDC abortion measure has a relatively small impact on the coefficients once the better abortion proxy is utilized, although the coefficient does rise by 20% in the regression with state-year interactions, which is likely to have the most measurement error. The bottom panel of Table 4 is identical in structure to the top panel, except that the dependent variable in the regression is the natural log of the number of property crime arrests rather than violent crime arrests. Using the better proxy once again leads the estimated impact of the abortion rate to more than double in all three specifications. Instrumenting with the CDC abortion measure has a substantial impact in column (3), nearly doubling the estimate to -.044, which is larger than any of the estimates for property crime originally reported in the flawed Table 7 of our 2001 paper. IV. Distinguishing between the alternative channels through which abortion operates The results in Table 4 above suggest that, properly measured, higher abortion rates when one is in utero is associated with a statistically significant and substantively large reduction in later crime. Abortion exposure can reduce aggregate crime in at least two ways: (1) by shrinking the size of the cohort, and (2) by lowering the average crime propensity of those who are born through positive selection. Neither Table 4 above nor Table 7 of Donohue and Levitt (2001) distinguishes (or claims to distinguish) between those two competing hypotheses. As Foote and Goetz 15 Note that this discussion refers to the impact of legalized abortion on crime, not the impact of legalized abortion on arrests. When analyzing the data by single year or age, we are forced to rely on arrest data rather than crime data, since the age of the offender is not known unless he is caught. As shown in Becker (1968), when crime falls, arrests are likely to fall less than proportionately. For the same size of the police force, with less crime there is more police manpower to devote to each crime, which should lead to a greater fraction of the crimes being solved by arrest. (2005) correctly point out, adding controls for cohort size to the regressions provides a mechanism for discriminating between the two avenues for abortion reducing crime. Foote and Goetz (2005) report the results from two specifications, one in which the natural log of the state population by single year of age is included as a control in the regression, and another in which the dependent variable is the natural log of arrests per capita, rather than simply the natural log of arrests. The latter specification is equivalent to imposing a coefficient of one on the population variable in the first specification. Once controls for population are included, any remaining impact of abortion is likely to be attributable to positive selection. Table 5 reports results from both of these specifications, along with regressions without controls for cohort size for purposes of comparison. The top row of the table, using the crude abortion proxy, replicates the results reported in Foote and Goetz (2005) for violent crime arrests. The second row shows identical specifications, but using the more carefully constructed abortion proxy. The third row uses the better abortion proxy and instruments with the CDC abortion measure. In stark contrast to the null results obtained by Foote and Goetz using the crude abortion proxy, with the better measure, the abortion rate remains negative and statistically significant, even with the inclusion of population controls or measuring arrests per capita. It is not a surprise that controlling for population reduces the magnitude of the estimates since a smaller cohort size is one channel through which legalized abortion plausibly reduces crime. These estimates suggest that at least 40 16 Foote and Goetz (2005) provide a discussion of the risks and benefits of these two specifications. If there are no social interactions, measurement error in the population variable, and that measurement error is uncorrelated with the other right-hand-side variables, then theory suggests that the latter of these two specifications is preferable. In the presence of social interactions or population measurement errors that are correlated with the right-hand-side variables, the former specification is probably more appropriate. 17 This discussion ignores any dynamic effects of abortion access on later cohorts. A woman who seeks an abortion today is likely to alter her future fertility decisions. See Ananat, Gruber, and Levine (2004) and Donohue, Grogger, and Levitt (2006) for an exploration of these issues. percent of the measured impact of abortion on arrests is operating through the selection channel. The bottom panel of Table 5 mirrors the top panel, except that it corresponds to property crime arrests. The results for property crime are weaker. The better abortion proxy by itself does not yield statistically significant estimates once population controls are included. In the instrumental variables regressions, however, the coefficient on abortion is -.028 (standard error=.011) when population is included as a control variable and -.013 (standard error=.010) when arrests are measured per capita. V. Conclusion We are indebted to Foote and Goetz (2005) for identifying a mistake in one of the tables in our original paper and pointing out the opportunity to more directly test the competing hypotheses regarding the explanation for why exposure to legalized abortion is associated with lower future crime. The inclusion of state-year interactions and controls for cohort size makes greater demands on the data by single year of age than the crude rule of thumb abortion proxy used in Donohue and Levitt (2001) could support. A more thoughtfully constructed proxy yields results that are in many cases stronger than those reported in our initial paper, even after addressing the issues raised by Foote and Goetz (2005). Thus, while criticism of us as authors for weaknesses in the initial paper are warranted, we do not believe that the Foote and Goetz analysis calls into question the conclusions reached in Donohue and Levitt (2001).